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Abstract  (Continued) 


The  following  points  will  be  demonstrated: 

#  Linear  regulator,  filter/observer  problems  can  be  solved 
using  linear  algebraic  equations  (in  distinction  to  solving 
nonlinear  Riccati  equations). 

•  ^he  R  weighting  matrix  need  not  be  positive  definite  nor 

is  it  necessary  thatvJjT^  exists.  l/fi: 


Singular  regulator  and  filter/observer  problems  (e.g.,  the 
cheap  control  problem)  can  be  handled  neatly. 

Hie  weighting  matrices,  R  and  Q,  can  be  explicit  functions 
of  frequency. 

There  are  some  advantages  in  using  non-diagonal  Q  and  R 
matrices.  _ 


In  addition,  a  new  class  of  problem  is  solved  using  the  Wiener-Hopf  approach. 
The  problem  treats  the  case  of  a  continuous  performance  index,  together  with 
a  continuous  plant  and  sampled  measurements  to  specify  both  the  optimal 
(discrete)  digital  control  law  and  the  optimal  (continuous)  data  hold  to  be 
used  for  coupling  the  digital  control  law  to  the  plant  actuators. 
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SECTION  I 
INTRODUCTION 


The  objective  of  the  research  effort  reported,  herein  was  to  develop,  via 
the  Wiener-Hopf  (W-H)  approach,  an  optimal  closed- loop  solution  for  digital 
control  of  continuous  plants  using  a  continuous  cost  function.  The  impor¬ 
tance  of  the  problem  class  resides  in  two  facts: 

•  Solution  consists  of  two  parts 

-  optimal  discrete  control  law 

—  optimal  continuous  data  hold 

•  Use  of  a  continuous  cost  function  assures  account¬ 
ability  for  the  inter-sample  behavior  of  the  continuous 
plant  response. 

This  is  worthy  of  further  elaboration.  The  use  of  a  continuous  cost 
function  in  conjunction  with  a  continuous  plant  model  yields  a  control  law 
that  is  optimal  at  all  instants  in  time.  However,  the  only  constraint  placed 
on  the  closed- loop  system  response  is  at  the  sampling  instances  when  a  dis¬ 
crete  cost  function  is  employed.  Hie  "inter-sample"  behavior  of  the  con¬ 
tinuous  plant  response  can  be  very  unsatisfactory.  This  is  especially  so  if 
the  data  rate  is  low  and  the  open- loop  plant  contains  lightly  damped  modes. 
Also,  when  a  discrete  cost  function  is  minimized,  the  data  hold  (the  coupler 
between  the  computer  and  the  control  actuators)  is  specified  arbitrarily  by 
the  designer  (more  often  than  not,  a  zero-order  data  hold  is  utilized) .  On 
the  other  hand,  if  a  continuous  cost  function  is  minimized  via  the  Wiener- 
Hopf  approach,  the  optimal  solution  specifies  both  the  control  law  and  the 
optimal  form  of  the  coupler. 

As  will  be  shown  in  Section  IV,  the  combined  coupler- control  law  problem 
yields  a  Wiener-Hopf  equation  that  is  more  formidable  than  its  discrete  or 
continuous  counterparts.  Therefore,  it  is  worthwhile  to  comment  on  the 
issues  involved  in  pursuing  a  solution  using  this  approach. 

The  advantages  of  using  modern  matrix  Wiener-Hopf  minimization  procedures 
are  not  widely  appreciated  in  today's  control  community.  The  primary  reason 
for  this  is  that  many  experienced  scientists  and  engineers  have  been  alienated 


TR-1 125-1 


1 


0NR-CR215-^60-1 


by  the  difficulty  in  applying  the  spectral  factorization  solution  method 
which  Wiener  used  (Ref.  l).  Very  few  optimal  control  practitioners  are 
cognizant  of  the  fact  that  Wiener-Hopf  equations  can  be  solved  by  a  direct 
solution  technique  which  makes  spectral  factorization  unnecessary  (Refs.  2 
and  3)  and  uses  mathematics  no  more  difficult  than  partial  fraction  expan¬ 
sions.  Moreover,  the  method  handles  the  multi- controller  cases  as  well  as 
unstable,  nonminimum  phase  plants.  We  believe  that  increased  awareness  of 
the  basic  and  simple  methods  available  for  solving  Wiener-Hopf  equations 
can  increase  modern  control  engineering  productivity  and  will  encourage  a 
more  integrated  use  of  time  domain  and  frequency  domain  techniques  for 
problem  solution. 

The  technical  development  proceeds  in  the  sections  which  follow.  Sec¬ 
tion  II  reviews  the  W-H  method  for  solving  the  continuous  regulator  problem. 

This  is  not  strictly  necessary  since  the  basic  solution  principles  for  the 
regulator  are  contained  in  Ref.  3.  However,  a  clear  understanding  of  the 
manner  in  which  the  direct  method  works  will  make  the  solution  method  used 
in  the  optimal  coupler  problem  easier  to  follow.  Moreover,  a  review  of  the 
regulator  case  affords  the  opportunity  to  make  clear  that  the  W-H  approach 
provides  a  unifying  framework  wherein  no  modifications  to  the  basic  approach 
are  required  in  order  to  treat  singular  cases.  In  particular,  the  "bulleted" 
items  in  the  abstract  will  be  clarified  using  a  series  of  numerically  tract¬ 
able  examples. 

Section  III  deals  with  the  optimal  linear  continuous  stochastic  control 
problem  via  the  Wiener-Hopf  formulation.  It  is  necessary  to  treat  this  case 
because  there  is  no  established  literature  on  solution  of  these  optimization 
problems  by  means  of  Wiener-Hopf  techniques.  First,  the  time  domain  form 
of  the  DUG  optimal  stochastic  control  problem  is  treated  using  frequency 
domain  methods.  It  is  shown  that  the  steady- state  gains  of  the  Kalman  filter 
can  be  found  using  the  W-H  approach  and  linear  solution  methods.  That  is, 
it  is  not  necessary  to  solve  a  nonlinear  Riccati  equation.  Next,  an  alter¬ 
native  W-H  formulation  is  postulated  which  does  not  make  use  of  the  separa¬ 
bility  principle.  This  leads  to  a  rather  interesting  formulation  in  that 
two  spectral  matrices  are  involved.  One  is  recognizable  as  being  the  regulator 
spectral  matrix,  while  the  other  is  recognized  as  the  spectral  matrix  associated 
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with  the  filter/observer  problem.  It  is  shown  that  this  W-H  formulation 
survives,  without  singularity,  the  limiting  condition  wherein  the  measure¬ 
ment  :'oise  vector  is  set  identically  equal  to  zero.  This  yields  the  optimal 
observer  solution  without  the  necessity  of  resorting  to  limiting  forms  or 
special  partitioning  (fo"  example.  Refs,  k  and  5). 

Section  IV  treats  the  optimal  coupler  probl-m  using  an  extension  of  the 
second  approach  discussed  in  Section  III  (i.e.,  the  formulation  which  does 
not  make  use  of  the  separability  principle).  Again,  there  will  be  two  spec¬ 
tral  matrices  which  appear  in  the  W-H  equation;  the  "regulator"  spectral 
matrix  remains  a  function  of  the  complex  frequency  variable  s,  but  now  the 
"filter  observer"  spectral  matrix  will  depend  on  the  delay  operator  z  =  esT. 

It  must  be  noted  that  the  situation  with  respect  to  the  design  of  a 
digital  controller  using  a  continuous  cost  function  will  suffer  the  same 
shortcomings  presently  associated  with  continuous  regulator  design.  That 
is,  questions  pertaining  to  the  selection  of  the  Q's  and  R's  of  the  regulator 
weighting  matrices  (so  as  to  produce  designs  which  are  not  only  "optimal" 
but  "satisfactory"  as  well)  will  obviously  persist  for  the  optimal  coupler 
problem.  In  this  regard,  it  is  hoped  that  the  examples  presented  herein 
(which  emphasize  the  use  of  non-diagonal  Q  and  R,  R  <  0,  etc.)  will  encourage 
optimal  control  practitioners  to  ease  self-imposed  restrictions  placed  on 
the  weighting  matrices. 
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SECTION  II 

LINEAR  REGULATOR  PROBLEM 

The  open-loop  plant  model  used  throughout  the  paper  will  be 

x  =  Fx  +  Gu 

X(s)  =  [Is  -  F]-1  GU  +  [Is  -  F]-1  x(o)  (l) 

=  AU  +  Bx(o) 

Assuming  a  control  law  of  the  form 

U  =  -KX  (2) 

results  in  the  closed- loop  configuration  shown  in  Fig.  1.  (K  is  not 
restricted  to  being  only  a  gain  matrix.)  The  continuous  cost  function: 

,  /■«+> 

J  =  2^3  J  j  (X*^  +  U*RU)  ds  (3) 

X#  =  X’(-s)  f  U#  =  U'(-s) 

is  minimized  by  taking  a  variation  on  U  such  that 

U  =  U0  +  XU,  (k) 

where  Uq  is  the  optimal  control  and  U,  is  any  physically  realizable  (exists 

for  t  >  0)  but  arbitrary  variation.  The  first  variation  gives  a  necessary 
condition  for  an  optimum  (refer  to  Ref.  3  for  the  details)  as: 
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Figure  1 .  Closed- Loop  Regulator 


Jc 


_  J  dS 


/’«{ 


(t)  t(t)  dt  =  0 


(5) 


where 

[R  +  A*QA]U0  +  A#qBx(0)  =  t  (6) 

Since  uj(t)  exists  for  t  >  0,  a  sufficient  condition  for  satisfying  Eq.  5 
js  that  i|r(t)  exists  only  for  t  <  0;  hence,  the  product  u^(t)  \|r(t)  is  iden¬ 
tically  zero. 

The  second  variation. 


Jd 


1 

2xJ 


+ 


A^A]U! 


ds 


(7) 


can  then  be  investigated  to  see  if  the  extremum  obtained  is  truly  a  minimum 
(Jd  >  0)  or  a  maximum  (J^  <  0).  Note  that  c  in  Eq.  7  is  not  necessarily  the 
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:i.‘uni'  ns  the  e  in  Kq.  ‘i.  Slnee  u’(t)  exists  I’nr  jk  1 1  i  v»  •  linn-,  its  lrfiplaee 
( t .wo- sided)  transform  exists  in  Uu<  s-plnne  in  a  domain  where  e  •  oj.  On  (In' 
oilier  htuui,  ij'(t)  exists  for  t  ^  0,  and  lia  trail.-; form  exists  in  t.lu*  plane  in 
a  domain  where  n  o...  Therefore,  e  (which  define::  the  pole::  enclosed  by  the 
contour)  muat.  aat.iafy  t.he  condition 

e  •  inln  o.„  o ,  (8) 

In  order  that  Uj  ^  and  ^  have  a  common  strip  of  convergence.  for  Kq.  *>  to  he 
Ident  ically  r.ero,  the  frequency  domain  requirement  la  to  close  a  contour  to 
the  loft  and  enclose  no  poles;  therefore,  the  stun  of  the  residues  (the  value 
of  the  Integral)  Is  r.ero.  The  path  traversed  parallel  t.o  the  Jm-uxls  is 
determined  by  the  value  of  e  In  Kq.  <‘t  and  Is  not  necessarily  the  Jaw  ax  is 
Itself  (e  m  ()). 

To  Insure  \|i(s)  exists  for  u  •  e.  (e.g.,  exists  in  some  left  half  plane), 
we  must,  pick  ll()  in  such  a  manner  that  any  poles  oi  Kq.  t«  whleh  exist  in  some 
right,  half  plane  (e.g.,  positive  time  t\met  ions)  cancel  Identically  into  each 
and  every  numerator  of  Kq.  (>. 

llo  will  have  the  form 


U0  W(s)x(o)  ('») 

The  required  compensation  can  be  computed  ns 

K  -W|  H  i  AW!_1  ( 10) 

For  the  special  ease  where  K  is  a  gain  matrix,  it  suffices  to  use  the  Initial 
value  theorem: 


K  ■  — sW(s)  III) 

s  — ■*“  i® 

Finally,  It.  Is  not  necessary  to  compute  the  e  hmt'd-  loop  transfer  fund  ions 
using 

XI s)  |  Is  -  K  i  dKf'x(o) 


TH- 11 1 


0NR-CR21 5-260-1 


since 


X  =  [B  +  AW]  x(o)  (l3) 

will  work  just  as  well.  In  fact,  Eq.  13  persists  even  in  the  singular  cases 
where  the  K  matrix  has  infinite  entries. 

What  are  the  candidate  poles  for  Uq?  These  are  picked  from  (Refs.  2,  3): 

P-1 

det[R  +  A  *QA]  £  (DC)  AS  (l4) 

where  P  is  the  number  of  controllers,  D  represents  the  open- loop  poles,  and 
A  represents  the  closed- loop  poles. 

What  are  the  candidate  zeros  of  Uq?  These  are  unknown.  Therefore,  one 
simply  specifies  polynomials  with  unknown  coefficients.  The  number  of 
unknown  coefficients  is  equal  to  the  number  of  "positive  time"  poles  which 
must  be  cancelled.  More  precise  details  are  available  in  Refs.  2  and  3;  our 
thrust  here  is  to  demonstrate  how  the  simple  principle  of  positive  pole 
cancellation  yields  the  solution  without  the  need  to  factorize  R  +  A*(JA. 

A.  SINGLE  CONTROL  POINT  EXAMPLE 

Let 


x  =  2x  +  3u 


so  that 


X(s)  =  x(°)  =  AU  +  Bx(0) 

S  c.  o  d 

Notice  that  the  unstable  open- loop  pole  exists  for  c  >  2.  Suppose  R  =  1 , 
Q  =  — 1/3  (Q  <  0),  then  the  W-H  equation 
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05) 


(15) 


Since  det[R  +  A*QAl  =  (— s  +  l)(s  +  1 ) ,  the  optimal  closed- loop  pole  is  at 
s  =  — 1 .  Let  Uq  =  |/A  =  |/(s  +  l)  and  substitute  into  Eq.  15: 


Only  one  positive  time  function  pole  (s  —  2)  survives  in  Eq.  l6;  therefore, 
only  one  unknown  coefficient  is  needed: 


(_s  f  l)aQ  _  x(o) 

(-S2  +  k)  =  * 

The  numerator  must  contain  (s  -  2)}  therefore,  the  numerator  equals  zero 
when  s  =  2. 

(-s  +  iJaJ  =  x(0)  =£  aQ  =  -x(o) 
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(17a) 

(17b) 

(17c) 


To  see  that  Jc  =  0,  consider  any  realizable  U-j  (not  necessarily  stable)  that 
exists  for  a  >  c-| .  The  Ui#  exists  for  a  <  — and  the  c  selected  is  the 
minimum  of  —2  or  .  This  is  sketched  in  Fig.  2.  Closing  the  contour  to 
the  left  using  c  <  min  a-^,—2  encloses  no  poles.  The  summation  of  the  residue 
is  zero,  and  the  value  of  the  first  variation  is  identically  zero.  Consider 
next  the  second  variation: 
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Jd 


Figure  2.  s-Plane  Domains 
1  ( c+j« 

=  271  I  ^.[R  +  A^U,  ds 
nJ  Jc-jco 

'/  AAV 

ds 


1 

r  c+>. 

u  5 

A  r,  \ 

II 

£1 

'c-joo  1 

Tu’) 

1 

2nd  d 

r  c+j~ 

rc-j«o 

F#F  ds 

■/; 


f2  dt  >  0 


08) 


An  interesting  observation  now  can  be  made  regarding  the  domains  of 
existence  when  the  open- loop  plant  is  unstable.  D  =  (s  —  2)  requires 
o  <  -2,  and  therefore  there  is  no  common  strip  of  convergence.  At  this 
point  two  options  are  available.  The  admissible  Ui  can  be  restricted  so 
that  AU^  /D  has  a  common  strip  of  convergence  with  Ui*S/D,  insuring  that 
Eq.  18  is  both  positive  and  finite .  On  the  other  hand,  the  only  real  issue 
is  whether  or  not  the  second  variation  is  positive.  Thus,  there  is  no  need 
to  restrict  Uj  if  one  is  willing  to  accept  a  positive,  if  unbounded ,  second 
variation . 
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B.  SIMPLIFIED  W-H  CONDITIONS 

It  is  not  necessary  to  work  with  the  complete  W-H  description  as  given 
in  Eq.  6.  Equation  6  can  be  reduced  to  two  computationally  simpler  require¬ 
ments. 

Since 


let 


det[R  +  A*QA]  e 


(dd)p”^  a  a 


u0  =  i(s)/A 


(19) 


(20) 


where  ?(s)  is  an  unknown  polynomial  matrix.  Further,  describe  the  plant 
matrices,  A  and  B,  in  terms  of  the  open-loop  poles  and  their  adjoint 
matrices.  That  is, 

A  a  Ba 

A  =  —  ,  B  =  -j-  ,  D  =  Open- loop  poles  (21 ) 


The  W-H  equation  becomes: 


[RDD  +  AaQAa]  5  ( s )  +  [AAaQBa]x(o) 
DDA 


* 


(22) 


From  the  previous  discussion  it  is  clear  that  each  and  every  numerator  of  ^ 
must  contain  D  and  A.  Therefore,  each  numerator  of  \|r  must  be  zero  for  those 
values  of  s  such  that  D  =  A  =  0.  (The  poles  of  D  may  be  located  anywhere  in 
the  s  plane.  It  is  the  fact  that  they  exist  in  some  half  plane  for  which 
0  >  ^1  which  "tags"  the  roots  of  D  as  giving  rise  to  positive  time  functions.) 
When  D  =0,  Eq.  22  reduces  to 


A*Q[Aa|(s)  +  Afiax(0)]  =  0  (25) 


TR-1 125-1 


11 


0NR-CR21 5-260-1 


Since  A$Q  is  already  analytic  in  some  left  half  plane,  it  need  not  be 
considered.  Thus,  we  have  a  first  W-H  condition. 

1st  W-H  Condition: 

Aa!(s)  +  ABax(o)  =0  ,  D  =  0  (2k) 

Next,  let  A  =  0  in  Eq.  22  and  obtain  the  second  W-H  condition: 

2nd  W-H  condition: 

[RDD  +  AaQAa]  6  ( s )  =  0  ,  A  =  0  (25) 

Notice  that  the  first  W-H  condition  persists  even  in  the  limiting  case 
of  Q  -»-0.  Furthermore,  even  though  Eqs.  2k  and  25  are  matrix  equations, 
it  usually  suffices  to  pick  only  one  component  of  each  since  Eq.  19  assures 
linear  dependence  when  D  =  A  =  0.  For  example,  using  the  first  component 
of  Eq.  2k  will  produce  the  same  linear  set  of  equations,  in  terms  of  the 
unknown  entires  of  f-(s),  as  will  the  second  (or  third)  component  of  Eq.  2k. 
Finally,  note  that  nowhere  does  R  1  appear  in  the  W-H  conditions  —  a  fact 
that  makes  the  direct  solution  method  ideal  for  the  evaluation  of  singular 
cases.  A  three-state,  two- control- point  example  will  be  used  to  clarify  the 
mathematical  details. 


Let 


0 

1 

0 

0 

1 

-5 

1 

5 

x  + 

—4 

2 

1 

0 

-1 

1 

0 

(26a) 
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X  =  AU  +  Bx(0) 


-4s+  1  (s  +  I)2 

s(-4s+l)  s(2s  - 3) 

S2  —  S  +  1  s  +  1 

s(s2  +  4) 


U(s) 


s2  —  1 

s+  1 

5 

-5s 

s(s  +  1  ) 

5s 

S-  1 

1  s2  —  s  +  5 

-  x(0)  (26b) 


Suppose 


0  0 

9k 

R  = 

,  Q  = 

0 

0  30 

-705 

(27) 


so  that  R  <  0,  Q  1  does  not  exist.  This  is  a  so-called  "cheap  control  prob¬ 
lem,"  since  there  is  no  weight  on  the  first  controller.  A  routine  but  tedious 
computation  gives: 


R  +  A#QA 


705s11  +  2209s2  j 

329s^  +  564  s2 

+  611  j 

+  846s  +  611 

-  - 

1  — 329s^  +  564s2  1 

30s° +  l46s^  j 

[_  -346s  +  611  j 

-37s 2 +  611  J 

DD 

(28) 

s(s2  +  4) 


Evaluating  the  determinant  of  R  +  A*QA  we  find 

det[R  +  A*QA]  oc  s2  +  2s  +  2  =  (s  +  I)2  +  (l)2 


(29) 

A  * 


♦Read  the  symbol  oc  as  "contains  the  factor". 
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Since  there  is  no  weight  on  U;,  we  find  that  the  order  of  the  closed-loop 
system  has  been  reduced  by  one.  In  order  to  force  both  D  and  A  to  cancel 
into  ijr  now  requires  only  5  unknown  coefficients  rather  than  6  (D  is  third 
order,  A  is  second  order).  Let 


U0  = 


=  W  x(0) 


(30) 


The  "burden"  of  the  extra  coefficient  is  assumed  by  the  first  controller, 
which  has  been  excluded  from  the  performance  index.  The  feedback  gains  from 
X  to  U  will  be  infinite,  since  U  is  not  proper-rational.  Consider  the  W-H 
condition;  only  one  component  of  each  matrix  need  be  considered.  Evaluate 
the  first  W-H  condition  for  D  =  0  and  obtain  three  equations  in  the  five 
unknowns.  Then  evaluate  the  second  W-H  condition  to  get  two  equations  in 
the  five  unknowns  (for  complex  roots  equate  real  to  real  and  imaginary  to 
imaginary) . 

1st  W-H  Condition; 

(-4s  +  l)(aos2  +  a-js  +  a^)  +  (s2  +2s  +  t)(bis  +  b2) 

-  -A[s2-1  j  S+  1  j  5]x(0)  (31) 

s  =  0,+2J  (d  *  o) 


2nd  W-H  Condition; 

(705s1*  +  2209s2  +  6ll)(aQS2  +  a,s  +  ag) 

+  (329s-*  +  564s2  +  846s  +  6l1)(bis +b2)  =  0 

s  =  -1  +  j  (A  =  o) 


(32) 
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We  obtain  five  equations  in  five  unknowns: 


s  =  2j,  Real 
s  =  2j ,  Iaag 

s  =  0 

S  =  -l  +3,  Real 
s  =-l  +  j,  Imaj 


(53a) 


(35b) 
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Placing  Eq.  35b  in  the  U  =  Wx(0)  format  gives: 


U  -  W(s)x(0) 

-1  36s2  -  425 s  !  - 1 8s2  -  60s  I  -327 s2  -  750s 
-289 


_1_  L 

255 


-79 9s  *  799 


-87 

-282  s -423 


-858 

-1128s-  1692 


(s2  +  2s  +  2) 


x(0) 


(34) 


There  will  be  three  infinite  feedback  gains  (refer  to  Eq.  11)  since  the 
first  controller  is  not  proper  rational. 


K 


.j  1 36ko  1 8ko 

255  79g  2q2 


(55) 


Note  that  the  first  controller  gains  go  to  infinity,  at  a  rate  determined  by 
the  ratios  of  the  entries  of  the  first  row.  Even  though  the  gains  are 
infinite,  the  closed-loop  transfer  functions  are  readily  computed: 


X  =  [B  +  AW]  x(0) 


255(a+l)  45  180 

544s-i309  327s  +  423  13089  +  1692 

-1368  -  34  -18s-  42  -72s- 168 


255(s2  +  2s  +  2) 


*i(0) 

Xg(0) 

xj(o) 


(36) 
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The  first  W-H  condition  guarantees  that  D,  the  open- loop  poles,  cancel  into 
each  numerator  of  [B  +  AW] . 

This  example  has  been  carried  through  in  careful  detail  and  demonstrates 
all  facets  of  the  direct  approach.  Other  examples,  which  derronstrate  the 
"bulleted"  items  in  the  abstract,  are  synopsized  in  the  appendix.  These 
examples  further  demonstrate  that  the  direct  solution  method  continues  to 
work,  without  need  for  modification,  regardless  of  the  form  of  R  and  Q. 


TR- 11 25-1 


17 


0NR-CR21 5-260-1 


8ECTI0N  III 

WIEHER-HOPF  FORMJLATION  —  OPTIMAL  LINEAR  STOCHASTIC  CONTROL 


The  open-loop  plant  equation  is  modified  by  the  addition  of  a  process 
noise  vector  n  and  an  output  equation: 


x  =  Fx+Gu+n  ,  x(0)  =  Xq 

y  =  Hx  +  v 


(37) 


The  transform  of  Eq.  37  is: 


X(s)  =  [Is  -  F]_1GU(s)  +  [Is  -  FI”1 [N  +  XqJ 

=  A(s)u(s)  +  B(s)[N(s)  +  Xq]  (38) 

The  block  diagram  of  the  open- loop  plant  is  shown  in  Fig.  3. 

The  time  domain  formulation  of  the  linear  optimal  stochastic  control 
problem  is  given  in  Fig.  4  (Ref.  6)  and  the  equivalent  frequency  domain 
formulation  is  given  in  Fig.  5. 


Figure  5.  Block  Diagram  for  the  Plant  and  Measurements 
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A  soparab  1 1  l  t.y  prlnoiplo  formulation  ir.  first  usod.  'Iliat.  is,  Uio  Kj 
matrix  roprosoiits  tlio  rogul.at.or  gain::,  and  now  tho  tank  in  to  find  tho 
Kalman  filt.or  gains  K;>.  Till:;  will  l>o  done'  by  minimising  t.lio  moan  a<|uaro 
orror  botwoon  t.lio  state  voot.or  X  and  i  t  a.  oat. inuit  o,  x.  Using  Fig.  aolvo 
for  I.  aiui  wrlt.o  )(  HI.: 

x'  nil  i  Kdinr1  K.v  i  iu  i  i  K  »m*  r 1  c:n  (vii 

lot. 


Wn  H|  I  t  KJIBl  1  K.  > 


()(0) 


and  writo  tho  orror  as 


K  X  -  X  Wfl|  v  i  HH(N  i  xt,l|  -  HTN  i  x0l  *  (-'ll) 

Forming  F„K  and  taking  t.lio  gradiont.  wit  It  roapoot  to  W;ll(  givos  tho  W-ll 
oquat.  Ion  (aoo  Kof.  or  rofor  to  Appotidlx  P): 

Wal‘%,  '  -  1^.11,11 ,  4  (•’»:') 

whoro  1«  t.lio  aut.o  spootra  of  t.lio  moaauromont  uoiao  and  la  t  ho 

auto  spootra  of  tho  proooss  no l so.  Tho  assumption  of  unoorrolatoil  N  and  V 
was  usoil  iu  arriving  at.  Kq.  It:',  although  t.lio  moro  gonoral  oaso  is  oasily 
t.roat.oil.  Furthermore,  x(o)  and  £(o)  havo  boon  sot  equal  to  /.oro  In  ordor 
t.o  simplify  tho  presentation.  To  oomponsat.o  for  this  shortcoming,  solut  ion 
of  tlio  output  regulator  protilom,  via  W-ll,  is  givon  tn  Appendix  P. 


«  -  X  W(lY  4  Bfl  I  KplIBr’GlI  ~  All  -  HfN  i  ;  A  IV. 

WnY  +  H[I  q  |T  -  I  -  K. jntlf.U  -  1l|N  t  x^,j 

WhY  -  WaHAU  -  B[N  +  xQ] 

-  Wnrv  •  ltB(N  •  x0 1  i  HAU  -  IIAUl  -  B' N  •  x,J 

Thoroforo. 

X  -  X  WjV  -I  im(N  ♦  Xq)!  -  BfN  <  Xol 

PO 
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Observe  that  now  the  unknown  of  the  W-H  equation,  Wa,  is  postmultiplied 
by  a  spectral  matrix  which  we  will  term  the  filter /observer  spectral  matrix. 
Equation  42  can  be  solved,  using  the  direct  solution  method,  in  exactly  the 
same  manner  as  the  regulator  problem  of  the  previous  section.  Moreover, 
once  Wa  is  found,  the  Kq  feedforward  matrix  can  be  computed  using 

K2  =  B_1Wa[I  -  HWa]_1 

A  simpler  result  is  obtainable,  if  one  considers  the  properties  of  Wa  as  a 
solution  to  a  W-H  equation.  Rewrite  Eq.  1+3  as 

Kg[l  -  HWa]  =  [Is  -  F]Wa  (44) 

and  observe  that,  when  Wa  is  a  proper  rational  function,  one  can  find  K2 
directly  by  letting  s  -*-00. 

*2  =  sWa(s)  (43) 

One  cannot  use  Eq.  45  in  the  singular  cases  which  occur  when  some  or  all 
of  the  measurement  noise  components  are  zero.  In  this  event,  (i  —  HWa) 
is  singular.  . 

Once  either  K2  or  Wa  is  known,  the  controller  to  measurable  output 
transfer  functions  can  be  computed  using  (refer  to  Fig.  5): 

U  =  — K-|[l  +  (I  —  WaH)AK-j  ]-1WaY  *  (46) 

WaY  -  B[I  +  KpHB ]~1GK1  X 

[I  +  B  (I  +  K2HB)~1GK1 ]-1  WaY 
[I  +  AK-j  -  BGK1  +  B(I  +  I^HB)-1  GK,]"1  WaY 
[I  +  AK^i  -  B( I  +  KgHB)”1^  +  KgHB  -  I)GK,  r'waY 
[I  +  AKi  -  WaHAK-|  ]WaY 
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or 

U  =  -K1  [Is  —  F  +  GK1  +  KgH]-1  K2Y  (47) 

Clearly,  Eq.  46  persists  even  when  the  feedforward  gain  matrix  K2  is  singular. 

Equation  46  gives  an  interesting  limiting  form.  Suppose  =  0  (no 

measurement  noise)  and  H  is  invertable.  Then  =  H-"1  satisfies  Eq.  42. 

But 

Wa  =  H-1  U  =  — K^Y  .  (48) 

The  following  illustrative  example  clarifies  the  mathematical  details. 

A.  ILLUSTRATIVE  EXAMPIE,  f  0 

Suppose  the  open-loop  plant  is  described  by 
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2  -1 


First,  solve  for  the  regulator  gainsy  call  this  matrix  Kj .  Using  the  direct 
methods  of  the  previous  section,  the  solution  of  the  regulator  W-H  equation 


gives 


U0  =  Wx(0)  =  - 


[s2  +  its+9  0  2s2-2  I 

2s2  +  3s  +  9  s2  +  5s  +  6  6s2  +  7s  +  32  J 


(s  +  1  )(s  +  2)(s  +  3) 


x(0)  (52) 


Application  of  the  initial  value  theorem  gives  the  feedback  gains  as 


Ki  = 


1  0  2 


2  1  6 


Further,  the  matrix  of  "regulator"  closed-loop  transfer  functions  is  computed 


X  =  [B  +  AW]  x(0) 


(s+  1 )  (s  +  3) 


— 6(  s  +  1 ) 


-5(s  +  3)  (s  +  2)(s  +  3)  s+32 


(s  +  1 )  (s  +  2) 


(s  +  l)(s  +  2)(s  +  3) 


To  differentiate  between  the  regulator  poles  and  filter /observer  poles, 
let  the  regulator  poles  be 


=  (s  +  1 ) ( s  +  2)(s  +  3) 
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The  second  step  is  to  find  the  Wiener  filter  Wa,  which  is  tantamount  to 
specifying  the  Kalman  gains,  Kg.  Hie  filter  portion  of  the  problem  has 
the  W-H  equation 


WaCcp^,  +  B#H#]  -  B<p^,  B*H*  =  *  (56) 


For  demonstration  purposes,  assume  only  one  output  measurement.  Let 


Y  =  HX  +  V 


H  -  [1,  0,  0] 


and  also  let  the  noise  sources  be  described  by  the  intensities 


=  1  *  ^ww*  = 


NN' 


85  0  0 

0  1690  0 

0  0  505 


A  straightforward  computation  gives 


<PVV'  +  HBtpNN'  B*H*  ~ 


^2^2 

DD 


where  (specifying  Ag  as  the  filter  poles) 

Ag  =  (s  +  4)(s  +  5)(s  +  6) 

Ag  *  — s5  +  15s2  -  74s  +  120 


(57) 


(58) 


(59) 


(60) 


Also,  one  finds 


BctNN'  = 


85s!t  -  1860s2  +  li‘,i*00 
-2115s5  +  1 4, 7^08 
85s5  +  2440a2  -  i»500s  +  1  4,  ’*00 
DD 


(61) 
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The  W-H  equation  has  the  form 


aQs2  +  alS  +  a2 
bQs2  +b-|S  +b2 
cQs2  +  c-|S  +  c2 
A2 


DD 


85s11  -  1 860s2  +  1  4,  400 
-21  15s3  +  1 4, 740s 
85s5  +  2440s2  -  4300s  + 1 4, 400 


Since  each  component  of  \jr  must  contain  D,  the  open-loop  roots,  we  require 
nine  unknown  coefficients  to  force  the  cancellation.  Of  course,  each 
component  can  be  treated  separately,  so  that  the  basic  problem  is  solving 
three  equations  for  three  unknowns,  rather  than  nine  equations  for  nine 
unknowns .  Letting  s  =  0,2j  gives  these  equations;  for  example, 


+  a^  + 


02 ^ ls=0,2j 


85s4  -  1860s2  +  14,400 
-.5  +  ^g2  _  +  120 


The  final  result  is 

15s2  +  70s  +  120 
70s2  +  60s 

13s2  +  2s  +  120 

-  - — 

a  (s +  2)(s  +  4)(s  +  5) 


(63) 
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Finding  the  Kalman  gains  is  new  a  simple  task,  since 


Once  either  Wa  or  Ko  is  known,  the  input /output  transfer  functions  can  be 
computed  using  Eqs.  46  or  47. 

Ms2  +  60s  +  19  I 

178s2  +  486s  +  701  I 

U  = - - - —  Y 

s5  +  21s2  -  7s  -  27 


Y  (65) 

It  comes  as  no  surprise  that  the  transfer  functions  describing  the  input/ 
output  relationships  between  the  controllers  and  measurable  outputs  may  well 
be  unstable,  as  they  are  in  this  example. 

Singular  cases  can  also  be  treated  using  the  direct  method  since  Eqs.  42 
and  46  can  always  be  solved. 

B.  SOLUTION  WITHOUT  THE  SEPARABILITY  PRINCIPLE 

The  linear  stochastic  optimal  control  problem  can  be  solved  without 
making  use  of  the  separability  principle.  This  can  be  very  useful  when: 

•  Hie  regulator  part  of  the  solution  is  singular 
(K,  — »). 
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•  It  is  desirable  to  reduce  the  computational  burden 
(e.g.,  if  one  has  a  very  high  order  system  but  only 
a  few  measurable  outputs  and  controllers,  it  is  more 
reasonable  to  compute  a  few  transfer  functions  than 
attempt  to  estimate  a  large  number  of  states). 

We  may  proceed  according  to  Fig.  6. 

In  Fig.  6  we  desire  to  minimize  the  performance  index 


J 


J_  fc+J" 

J c-> 


i*'*irnn~innniYii->i->  ii , 

(X#QX  +  U#RU)  ds 


(66) 


where  ( • )  denotes  the  expected  value  of  ( • ) . 

First,  form  the  integrand  of  the  performance  index 


•  =  X#QX  +  IJ#RU 


(67) 


using  the  equations  (refer  to  Fig.  6) 


U  =  -[I+WHA]-1  W[V+HBN]  (68) 

X  =  BN  +  AU  =  BN  -  A[I +WHA]_1  W[V+ HBN]  (69) 

Note,  from  Eq.  68,  that  the  closed-loop  stability  is  determined  by 

Wa  =  -[I  +WHA]~1  W  *  (70) 


*2he  W  matrix  of  Fig.  6  and,  for  example,  Eq.  70,  is  not  the  W  matrix 
of  the  regulator  equation  U  =  Wx(0). 
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Figure  6.  Direct  Solution  of  the  Linear  Stochastic 
Optimal  Control  Problem* 

Thus,  we  can  minimize  Eq.  66  with  respect  to  Wa  and  assure  closed- loop 
stability.  The  W-H  equation  becomes  (using  the  variational  method  of 
Ref.  3  or  see  Appendix  C). 

[R  +  A#QA]Wa[cp^,  +  H  B  qjjfo.B.H*]  (71 ) 

+  A*Q  B  cp^,B#H#  = 

for  the  special  case  where  N  and  V  are  independent.  Therefore,  solve  the  W-H 
equation  for  Wa  and  compute  W  using  Eq.  70,  i.e., 

W  =  -Wat I  +  HAWa]_1  (72) 

The  separability  principle  is  still  very  much  in  evidence  in  Eq.  71,  since 
[R  +  A#QA]  determines  that  group  of  closed- loop  poles  which  correspond  to 
the  optimal  regulator  solution,  while  +  HBqp^,(H  B)'  determines  the 

*The  W  matrix  of  Fig.  6  and,  for  example,  Eq.  70,  is  not  the  W  matrix 
of  the  regulator  equation  U  ■  Wx(o). 
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remaining  closed- loop  poles  which  correspond  to  the  optimal  filter /observer 
solution.  Equation  71  can  also  be  solved  using  the  algebraic  methods  pre¬ 
viously  discussed.* 

Application  of  this  approach  gives  the  direct  relationship  of  the 
controller  to  the  output  since 

U  =  -WY  (73) 

An  effect  of  formulating  the  problem  in  this  way  can  be  a  dramatic  reduc¬ 
tion  in  dimensionality  of  the  problem  solution.  Application  to  the  illus¬ 
trative  example  given  in  the  previous  subsection  yields  the  exact  same 
transfer  function  W  with  less  numerical  detail  (i.e.,  the  solution  is  the 
same  as  Eq.  65.) 


♦In  addition,  a  spectral  factorization  algorithm  is  given  in  Appendix  B. 
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SECTION  IV 

THE  OPTIMUM  COUPLER  PROBLEM 


The  prime  objective  of  the  research  effort  reported  herein  was  to 
develop  an  optimal  closed- loop  solution  for  direct  digital  control  of  con¬ 
tinuous  plants  using  a  continuous  cost  function.  As  noted  in  the  introduc¬ 
tion,  the  solution  consists  of  two  parts  —  an  optimal  discrete  control  law 
and  the  optimal  (continuous)  data  holds.  We  have  elected  to  proceed  with 
the  development  by  extending  the  second  method  of  the  previous  section  in 
a  manner  which  accounts  for  sampled  output  signals  which  are  to  be  processed 
by  a  digital  computer  and  outputted  through  a  data  hold  (coupler)  to  the 
control  actuators.  In  the  case  of  the  continuous  filter /observer  problem 
two  spectral  matrices  came  into  playj  the  matrix  which  pre-mult iplied  the 
unknown  of  the  W-H  equation  was  recognized  as  the  regulator  spectral  matrix, 
whereas  the  post-multiplier  was  recognized  as  the  filter /observer  spectral 
matrix.  Both  were,  in  the  illustrative  example,  rational  functions  of  s. 

In  the  W-H  equation  for  the  optimum  coupler  case,  it  will  be  seen  that  the 

pre-multiplier  remains  a  function  of  s  while  the  post-multiplier  becomes  a 

sT 

function  of  the  delay  operator  z  =  e 

The  situation  of  interest  is  depicted  in  Fig.  7.  For  brevity,  N  +  x(o) 
is  taken  as  N.  That  is,  the  initial  condition  input  vector  will  be  suppressed. 


Figure  7-  Linear  Stochastic  Optimal  Discrete  Control 
of  a  Continuous  Plant 
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Let  the  integrand  of  the  performance  index,  call  it  4>,  be  the  expected  value 
of  the  usual  quadratic  index: 


$  = 


X*QX 


+  U*RU 


(74) 


First  develop  expressions  for  the  continuous  X  and  U  (the  superscript  T 
will  be  used  to  denote  that  a  signal  is  impulse  sampled  at  l/T  samples/ 
second;  a  slash  will  be  used  to  denote  the  transpose): 

U  =  — Muf  =  — M[l  +  WT(HAM)T]-1WT[V  +  HBN]T  (75) 

X  =  — AM[I  +  WT(HAM)T]_1WT[V  +  HBN]T  +  BN  (76) 


We  choose  to  optimize  with  respect  to  the  matrix  of  transfer  functions 
between  [V  +  HBN]t  and  U^j  therefore,  let 


Wa  =  ~[I  +  WT(HAM)T]-1  WT  (77) 

and  define 

It  =  V  +  HBN  (78) 

l2  s  BN  (79) 

so  that  Eqs.  75  and  76  become 

U  =  (80) 

X  =  amw^t  +  i2  (8l) 


Substituting  Eqs.  80  and  81  into  Eq.  74  gives  the  integrand  of  the  perfor¬ 
mance  index 

*  =  RtMwJif] 

+  +  l2]  (82) 
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Next,  take  the  gradient  of  Eq.  82  with  respect  to  the  unknowns  MWa,  take 
the  expectation  and  arrive  at  the  W-H  equation: 

[R  +  A^QA]Mw|<p' (i^  )  +  A#Qfp'(S^!2)  =  t  (85) 


Assuming  V  and  N  to  be  independent  noise  processes  gives  (see  Fig.  8): 


9  m  T, 

*1*1 


T 


+  HB(Pnn»B*H*  ' 


m 


(85) 


The  i/T  scale  factor  is  in  keeping  with  the  definitions  given  in  Chapter  10 
of  Ref.  7.*  In  a  like  manner,  the  cross  spectra  between  the  sampled  vector 
and  the  continuous  vector  simply  the  scaled  continuous  spectra 

(Ref.  7): 


(prr6  “  t  Bcpnn'b*h* 
12 


(86) 


Figure  8.  Spectral  Noise  Model 


*The  use  of  l/T  in  Ref.  7  appears  to  be  a  "matter  of  convenience." 
Other  reference  sources  (e.g..  Ref.  8)  do  not  use  it. 
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The  W-H  equation  now  takes  the  form 


[S  -  A„<«  MW l  [4-  ♦  4 


*  T  -  * 


A.  SUMMARY  OF  OPTIMAL  COUPLER  PROBLEM 

Given  the  digitally  controlled  continuous  system  of  Fig.  9,  the  W-H 
equation,  resulting  from  minimizing  the  expected  value  of  a  quadratic  index. 


R 

♦  a.qakwI)  [-f1  *  4  (h%k'b*h*)T. 


where  w£  is  defined  as 


-[I  +  wt(ham)t]  1  WT 


u,  -  u  — 

|— *■  M  — ►  A 

DATA 

HOLD 


Figure  9.  Linear  Stochastic  Optimal  Discrete 
Control  of  a  Continuous  Plant 
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¥  *  7 


-  ( ■  !P  -  f 

T  \-s2  +  -j  f 


j_  J5  _J _ i_ 

T2  2T  s  +  1  s  -  1 


The  power  spectra  of  sampled  white  noise  has  been  taken  as  l/T‘"  rather  than 
1 /T  (see  Ref.  7  for  a  discussion).  Continuing,  we  find 


±  +  n  r  «  _ 

12  rTb-eT  z~eTJ 


4  w-2- 


m  Z(e-T_ 


T-  ( z  —  e—i ) ( : 


" 

-e  ) 

T7 
-e  ) 


afcifLlL 
12  L  (*-e'T 


--tt  T(e  r-  eT)|z  +  1 


)(z-eT) 


_L  (2~e  BT)(z-eBT) 
T‘  (z-e  r)(z-eT) 


Thus,  the  optimal  filter/observer  pole  is  defined  by  z  -  c  ;  the  value 
of  B  as  a  function  of  T  is  given  in  Table  1 


TR- 11 25-1 


0NR-CR21 5-260-1 


TABLE  1 


The  only  remaining  computation  is 


3.9998 

3-9769 

3.0285 


T  A*QBaNN'B*H*  "  T(_s  +  ,)2(s+  1) 


The  W-H  equation  is  therefore 


(-a  +  3)(s  +  3}  MUT  U-e~BT)(?-eB^ 


(-s+  1)(s+T)  a  T2(z_e'JT)(z-eT) 


T(-8  +  1  )2(b  +  1) 


The  application  of  the  direct  approach  requires  that  MW£  be  such  that  the 
numerator  of  *  cancels  all  those  poles  which  can  produce  positive  time 
functions  —  that  is,  cancel  the  terms  s  +  1  and  z  -  e  .  A  selection 


,  -Tv 

-  e  ) 

NWj  =  - _T»m  f 

a  (s  +  3)(z-eBT) 


(100) 
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where  is  an  undetermined  coefficient,  is  sufficient  to  achieve  this  goal. 
Substitution  of  Eq.  100  into  Eq.  99  gives 

(-s  +  3)(s  +  3)  a0^z~e~T) 

(— s  +  1 )  ( s  +  1 )  (s  +  3)(z_e-BT) 

x  (z-e~BT)(z-cBT)  +  120T 

(z-e-Jr)(z-eT)  (— s+l)^(s+l) 

or 

(-s  +  l)(-s  +  3)a0(z-eBT)  +  120T(z-eT) 

(— s  +  l)2(s  +  l)(z  — eT) 

In  Eq.  102,  must  be  selected  so  that  the  numerator  is  zero  when  s  =  -1 ; 
therefore. 


(101) 


(102) 


®0 


120T(e~T~eT) 

2(M(e~T-eBT) 


-15T 


H-H) 

(1+B)T 


[1-e 


] 


For  example,  when  T  =  1  sec. 


-1 • 736929333 ( z  -  e~T) 
(s  +  3)(z-e~BT) 


T  =  1.0 


(103) 


(101*) 


Notice  the  general  result.  'Hie  data-hold  poles  are  defined  by  the  (con¬ 
tinuous)  regulator  spectral  matrix,  while  the  "discrete"  poles  of  sure 
determined  by  the  sampled  filter/observer  spectral  matrix.  Letting 


M 


1 

8  +  3  ' 


a0(z-e-T) 
(z- 


(105) 
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one  then  proceeds  to  compute  the 


digital  control  law  using  the  equation 


First, 


wT  =  -w£[i  +  (ham)tw|]  1 


J_  (e-T-e-^T)2 
2  (z  —  e- T)(z  —  e  3T) 


Therefore, 


(ham)tw|  = 


JL  ( e~T -  e  3T)Z 

2  U^Ms-e-31)  (z-e'BT) 


and 

i  +  (ham)tw| 


(106) 


(107) 


ii 


U-«r3T)U-e'BT) 


2  +  B 


( 108) 


That  is,  as  a  consequence  of  the  W-H  process, 

— T 

contains  the  "open- loop  root,"  z  -  e  ,  as  an 


the  numerator  of  I  +  (HAM)^W^ 
exact  factor.  Therefore, 
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WT  .  -apCz-e^) 

z -  e-BT  (z-e“aT)(z-e_T) 

-a0(z-e  3T) 


(109) 


It  is  interesting  to  note  that  the  output  equation 


U  =  -WT  Yt 


(110) 


can  be  written  as 


=  -MWTYT 


(HI) 


That  is,  the  data  hold  can  be  viewed  in  a  manner  quite  similar  to  the  zero 
order  hold  (see  Fig.  10).  It  is  now  apparent  that  the  optimal  coupler 
solution  defines  a  data  hold  which  forces  the  plant  to  follow  a  path,  during 
the  intersample  period,  that  is  in  a  sense  "scheduled"  by  the  continuous 
constraints  placed  on  the  solution  by  the  (continuous)  spectral  matrix 
R  +  A*QA. 


TR-1 125-1 


39 


0NK-CR21 5-260-1 


Figure  10.  Impulse  Response  of  Zero-Order  Hold 
and  Optimal  Coupler 

The  example  has  focused  on  the  numerical  details  of  solving  the  W-H 
equation  when  the  regulator  spectral  matrix  is  a  function  of  s  and  the 
filter /observer  spectral  matrix  is  a  function  of  z.  In  the  next  section 
we  treat  a  "limiting  form"  problem  in  order  to  focus  on  the  properties  of 
the  optimal  coupler  (in  distinction  to  W-H  solution  techniques). 

C.  THE  OPTIMAL  REGULATOR / COUPLER  PR0BI£M 

We  return  now  to  the  notation  of  Section  II  and  treat  the  optimal 
coupler  for  the  special  case  where  N  and  V  are  zero  —  the  only  excitation 
being  the  initial  conditions.  Recall,  for  the  optimal  regulator,  the 
Wiener-Hopf  approach  solves  for  the  optimal  controller.  Therefore,  it 
does  not  distinguish  between  the  two  configurations  shown  in  Fig.  11. 

Given  a  performance  index  for  which  the  integrand  is 


<t>  =  X^QJC  +  U*RU 


(112 
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Whereas  for  Fig.  11b  the  compensation  and  data  holds  are  defined  by: 


MKT  =  -W  |[B  +  AW]1}"1 


(116) 


If  a  discrete  index  is  used, 


4>  =  X^QXT  +  u£RUT  (117) 

then  the  W-H  equation  is  (M  given) 


[R  +  (AM)J  Q(AM)T]  UT  +  (AM)^QBTx(0)  =  f 


(118) 


The  solution  is 


UT  =  WTx(0)  (119) 

The  gains  can  be  found  with  the  (z -domain)  initial  value  theorem. 

Clearly,  for  a  problem  formulation  which  is  as  simple  as  the  regulator, 
the  use  of  a  continuous  index  forces  the  same  smooth  continuous  motions 
in  the  state  and  controller  deflections  for  the  discretely  controlled  case 
as  it  does  for  the  analog  controller.  The  situation  is  not  nearly  as  simple 
to  predict  when  the  discrete  index  is  used,  because  the  migration  of  z-plane 
"zeros,  as  a  function  of  q  and  r,  is  usually  not  very  transparent. 

We  will  solve  the  W-H  equations  and  give  transient  responses  to  demon¬ 
strate  these  points.  Specifically,  three  cases  are  considered; 

1)  Continuous  index,  continuous  controller. 

2)  Continuous  index,  discrete  controller,  data  holds 
specified  by  W-H  solution. 

3)  Discrete  index,  discrete  controller,  data  holds 
specified  a  priori. 
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OPEN-LOOP  PLANT 


•  +  >]'*[>]• 


(120) 


so  that 


s  +  1  £ 

5s  +  1 


5s  +  1  -4  s  + 1  I 

k_ _ iu  +  — - —  x(0)  =  AU  +  Bx(0) 

s(s  -  3)  s(s  -  3) 


(121) 


Suppose 


=  1  (a  scalar)  ,  Q  = 


(122) 


R+A^QA.  = 


(s2  +  1.2s  +  . 2)  (s2- 1.2s -.2)  =  AA 

'  s(s  —  3) (  s)(—  s  -  3)  r>  n 


(123) 


Since  there  is  only  one  controller,  exercising  the  first  W-H  condition 


gives 


U  =  „(.)„«,)  -  I-8(,;’)  ! 

s2  +  1 .2s  +  .2 


r  .8  1  -1 1 

s+  .2  i  s  +  1  J 


(124) 
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The  matrix  of  closed -loop  transfer  functions  is: 


1 

s  +  .2 


0 


1 

+  1 


x(0) 


(125) 


The  feedback  gains  can  be  computed  using  either  the  initial  value  theorem 
or  Eq.  115: 


K  =  [  -.8  ;  1  ] 


(126) 


As  noted  previously.  Case  2  must  have  the  same  W-H  equation,  and  therefore 
the  same  solution: 


U0  =  W(s)x(0) 


(127) 


The  compensation  (data  holds  and  gains)  is  defined  by  Eq.  11 6: 


MKT  =  -W 


j[B  +  AW]T| 


r’  -  - 

.8 

1 

z-e"-2T 

V 

i  - 

s  +  .2 

— 

1  s  +  1 

n 

z 

z  -e 


-T 


[=18Ii=£^£]  ! 

“I  s+.2  [  s  +  1  J 


(128) 


bb 
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That  is. 


M  = 


1  _  -T(s+.2) 


s  +  .2 


!  !_e-r(s+i) 
;  —  — 


One  may  verify,  hy  direct  substitution  of  Eq.  127,  that 


U  =  -M[I  +  KT(AM)T]  1KTBTx(0)  s  W(s)  x(0) 


and  therefore 


(129) 


X  =  (B  +  AW)  x(0) 

Next,  it  is  a  straightforward  exercise  in  discrete  regulator  theory  to 

-JP  — 

obtain  the  Case  3  result  [q's  and  r's  were  used  which  forced  (z  —  e  )(z  —  e  * 
as  the  closed-loop  poles].  The  gains  that  achieve  this,  as  a  function  of  the 
frame  time  T,  are  given  in  Table  2  (u  =  -KXT  =  -[Ki  K2][Xi  X2]T)- 


TABLE  2 


T  =  .1 

T  =  .5 

11 

«1 

792556055 

-.771538772 

-.758184683 

k2 

.95^136595 

.836065309 

.776195811 

The  discrete  controller  deflections,  as  a  function  of  x(0),  can  be  computed 
using 


UT  =  -n  +  K(AM)T]  1KBTx(0)  (132) 

and  the  continuous  state  motions  using 


X  =  Bx(0)  +  AMUT 


(133) 
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(in  computing  Eq.  133  it  is  best  to  carry  the  analysis  to  a  point  -where 
the  strongly  unstable  open-loop  root  has  been  cancelled  out.)  The  tran¬ 
sient  responses,  for  T  =  1,  x.|(o)  =  1,  ^(O)  =  4  are  shown  in  Pig.  12. 

Note  the  poor  transient  response  of  the  system  designed  with  discrete 
regulator  theory.  Whereas  the  continuous  design  was  "decoupled"  as  fax  as 
the  initial  condition  response  (each  state  containing  only  one  mode),  the 
discrete  design  forced  the  correct  poles,  but  the  zeros  go  where  they  will. 
This  can  be  verified  by  checking  the  XT  closed-loop  transfer  functions: 


X 


T 


z-.  3699121 7 
.024976780 


.0365270 
z- . 81669801 


(z-e  *2T)(z-e-T) 


X1  (0) 

x2(0) 


(134) 


t(sec) 


Figure  12.  Comparison  of  Transient  Responses 
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To  summarize,  the  optimal  coupler  solution  forces  the  same  initial  condi¬ 
tion  response  as  associated  with  the  continuous  regulator  design.  The  discret 
regulator  design  exhibits  undesirable  inter-sample  response.  Indeed,  with 
only  two  gains,  the  closed  loop  poles  can  be  specified  but  the  zeroes  go 
where  they  will  —  resulting  in  a  design  which  cannot  even  match  the  continu¬ 
ous  response  at  the  sampling  instants. 
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SECTION  V 
SUMMARY 


A  now  class  of  problem  la  solved  using  the  W-H  approach,  Hie  problem 
treats  the  ease  of  a  eontinuoua  performance  Index,  together  with  a  coti- 
tlnuoua  plant  and  sampled  measurements  to  specify  both  the  optimal  (discrete) 
digital  control  law  and  the  optimal  (continuous!  data  holds  to  bo  used  for 
coupling  the  digital  control  law  to  the  plant-  actuators.  The  optimal  coupler 
orchestrates  an  inter-sample  controller  response  leading  t.o  dosed  loop 
systems  which  exhibit  characteristics  markedly  superior  to  the Jr  discrete 
regulator  counterparts , 

There'  are  several  important  circumstances  in  which  direct  digital  design 
with  respect  to  a  continuous  cost  tliuctiou  is  crucial  to  efficient,  and  suc¬ 
cessful  synthesis  of  digital  control  laws.  Those  circumstances  are: 

•  Existing  systems  for  which  the  limits  on  con', 
word  length,  and  computational  ape  eel  hnv  '  bevn 
rear  he'd  (for  example,  V/STOIANP). 

•  Applications  whore  onto”  loops  are  updated  at. 
inherently  slow  rates  [for  example',  lUTi-td’,  MUl, 

NASA's  remotely  piloteil  ivst'arch  vehicle  (RlllV)J. 

•  Applications  wherein  the  e'ost  of  a  fast,  minicomputer 
is  not.  Justifiable  but.  a  microprocessor  is  cost, 
effective  (for  example,  general  aviation,  digital 
controllers  for  missiles,  RPV's  and  other  smart, 
expendable  weapons ) . 

•  Application  of  low  data  rate  techniques  for  real- 
time  simulation  problems. 

In  ueidltion,  the  developments  of  Sections  IT  and  111  make  clear  the  uni¬ 
form  manner  in  which  the  direct  solution  method  treats  singular  problems. 

That  is,  no  modification  is  required  to  solve  cases  for  which  some  (or  all) 
of  the  feedback  regulator  gains  or  some  (or  all)  of  the  feedforward  gains 
of  the  filter/observer  go  to  infinity. 
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Furthermore,  the  examples  of  Appendix  A  demonstrate: 

•  The  R  weighting  matrix  need- not  be  positive  definite 
nor  is  it  necessary  that  R“'  exists.  Q  can  be  £  0. 

•  Singular  regulator  and  filter/observer  problems  (e.g. 
the  cheap  control  problem)  can  be  handled  neatly. 

•  The  weighting  matrices,  R  and  Q,  can  be  explicit  func 
tions  of  frequency. 

•  There  are  advantages  in  using  non-diagonal  Q  and  R 
matrices . 
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APPENDIX  A 

ILLUSTRATIVE  REGULATOR  EXAMPLES 

Illustrative  regulator  examples,  designed  to  demonstrate  the  "bulleted" 
items  of  the  abstract,  are  synopsized  in  this  appendix.  In  general,  we  list 

•  The  open  loop  plant}  x  =  Fx  +  Gu 

•  The  transform  of  the  open-loop  plant}  x  =  AU  +  Bx(0) 

•  The  weighing  matrices}  R,  Q 

•  Wiener-Hopf  equation  and  the  assumed  form  for  the 
optimal  control 

•  First  W-H  condition 

•  Second  W-H  condition 

•  Solution  for  the  unknowns  of  Uo 

•  Optimal  control  gains 

•  Matrix  of  closed- loop  transfer  functions 
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EXAMPLE  —  UNSTABLE  OPEN-LOOP  PLANT,  SINGLE  CONTROLLER 


M 


EXAMPLE  —  i,R  MAY  B£  FUHCTrOHS  OF  FREQUENCY 


Observation 

Lead/lag  compensation 
Open- Poop  Plant 


x  = 


Weighting  Hatriees 
-s2  -  51 12 


R  = 


-  ur-  *  9 


a  = 


[973-*-+  o  "j 

0  -4336  J 


det[R*.A.5\]  =  (s*4)(s2*27a-?34) 

x  (-s  *  t)(s?-?7s  *  23*) 


W-H  Equation 

[R*A,^\]Uo*.A,q3x(0)  =  (- 

E-H  Requirements 

A  -  (s  +  4)(s?  +  27s*234) 

Solutions  for  Untaowna 
a  =  o 

»2  -  [-936  -936]  x(o) 


A1(a^a2<'a1s-*ag)  *A(a?-9)[9734i*(s-  1)  j  -S3*' s'  -  973-^] 

D5i(a-3)(-s*3)  ~*f'~ 


*(0)  = 


SqsS  *  8,8  ♦  a, 
A  ' 


8  ■  -t 

9ao  -  3a,  *•  ag  =  o 


*  numerator  =  o  when  =  o,  s  =  -3 


s  .  -3 

a0_a1ts2  5  [2  -624]  x(C) 


9  -3  1 

1  -1  1 

— 

*0 

a1 

- 

0 

0 

0 

-624 

=*> 

“  • 

®c 

-  312 

-1243 

0 

-312 

0  0  1 

*2 

-936 

-936 

«2 

-  936 

-936 

. 

Optimal  Control  Compensation 

U  =  Wx(0)  =  [-(31SS-*- 1243s  *930  j  -(3l2s*95c)1  .  , 

U  *  4)^*278*234) - 2 -  *(0) 

»  dj’g(8*  5)(s  *  1)  !  3i?fs*5)1 

(a  +  4)(s-  ♦  27s  *  234)  x(0) 

i-tatrlx  of  Closed-Loon  Transfer  Functions 

[a?  *  31a  ♦  30  s  ♦  30  "J 
-31?(s*3)  sf  a  *  3o) | 


*H3 

p— 3 — 

I - 0- 


X  =  -W[B  *  AV.’] 


■  [  W  ;  -•] 


x  *  [B*AW]x(0) 


_ _  x,(0) 

(s  +  4)(a^  +  278  *  234)  |^,(0) 


r- 


] 


A-U 
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EXAMPLE  —  ONE  -CHEAP’’  CONTROL 


Open- Loop  Plant 


0  1  0 

o 

t _ 

X  ■ 

-5  1  5 

X  + 

—k  2| 

r ” 

o 

1 

L~* 

,  o 

U  X(s)  = 


-La  +  1 

(a+l)2' 

82  -  1 

S  -f  1 

5 

*H+a  +  1) 

s(2s  -3) 

-5« 

s(s  +  1 ) 

5a 

.  s2  -a  +  1 

a  +  1  . 

•  u  +  ■ 

.  a-  1 

1 

32  -  8  +  5. 

a(s2 

+  M 

a(s2  +  U) 

x(O) 


=  Au  +  Bx(O) 


Weighting  Matrices 


0  0  91* 

Q  »  0 

o  30  -705 


W-H  Equation 

[R+A.3A]Uo  +  A.C}Bx(0)  =  * 


U0  ■  Wx(0) 


det[R  + A„3A]  oc  s2  +  2b  +  2  =  (s+l)2+(l)2 

[»0«2  *  *1«  +  »2  I 
_ b? _ 1 


a2  +  2a  +  2 


Note:  Place  the  "burden"  of  the  extra  coefficient  on  the  controller  which  is  excluded  from  the 
performance  index. 

1st  W-H  Condition 

( — Ua  +  1)(aQS2  +  a,#  +  ag)  +  (a2  +  2a  +  l)(bja  +b2)  «  -&( a2 -  1  a+1  5]x(0)  a  »  0,  +2J 


2nd  W-H  Condition 


EXAMPLE  —  R  a  0,  1W0  "C1EAF"  CONTROLLERS 

Cbaervatlon 

W-H  approach  require*  no  Bodification  when  R  =  0. 

Open- Loop  Plant  f  -4.  -  I  U.  I 


[o  1  ol  To  ll 

■5  1  5  I*  *  I-4  2|' 

1  0  -lj  [_1  oj 


r  -4a*  1 

{*  ♦  I)2' 

* -  - 1  a  ♦  1  5 

la(-4a  -  1) 

*(2a-3) 

► 

•C 

1 

ru 

«n 

a  *  1  . 

II  -  . 

.8—1  1 

•(a2  ♦ 

4) 

s(a?*4) 

Au  +  Bx(0) 


Weighting  Matricea 


0  0 
0  0 


1  0  0 

0  -3/2  0 

0  0-5 


det[R*A#2A}  a:  [so)  A  reduced  by  two 


l  J  L°  o 

M  Equation  F  a,a  -  am  1 

[R*  A,qA]'J0  *  A.TBx(O)  »  *  U0  -  Wx(0)  =  L—  -bg  i. 

A 

lat  W-H  Condition 

(-*«  ♦  l)(ara  »ag)  ♦  (a  ♦  l)2(b,a*b2)  =  -d[a?-l  |«+1  j  5]*(0)  when  a  •=  0,  *2j 

2nd  v;-H  Condition 

where 

NA(ai*  ♦  *e)  ♦NB(b1*  *b2)  »  0  »A  *  <li  C— 16»2  +  1)  +  q2(-a2)(-l6a2  ♦  1)  ♦  qjfa1*  ♦  a2  ♦  l) 

when  a  *  -1  Kg  -  q,(4a^  +  9*‘  *  6a  ♦  1)  *  qgf-fla1*  ♦  10a3  ♦  3*2)  .  qj(*3  -  2a?  .  2a  ♦  1 ) 


when  a  *  -1 

Solution  for  Unknown* 


16  1-8-3  a, 

2  —8  —6  4  eg 

0101  b) 
7-7-3  3  bj> 


ptlaal  Control  Oalna 
U  -  W(a)  x(0) 


5  3-5 

10  -4  -10 

1-1-5 
0  0  0 


-18  9  10 

12-6-50 
-56  15  60  x,(0) 

■ii  -?--*> J  x?(0) 

56 


[rL0f  *_I2  J  2*~ 6  i.ioa-5o"|  r  ,0l  *  "7  , 

-56a  ♦  14  1 15a  -20  tOa"-8o  J  l  I'8*  ~  9*  “'0*  I 

-  55  — - ,  ~  1 -  *2'°'  ■  3?  I  ,  I 

(«*D  ,,(0)  (_  56a  -15a  -60a  J  Llm 

Even  though  the  gain*  go  to  infinity,  they  eiuat  do  ao  with  the  ratioa  defined  by  the  equation  for 
K  and  the  reault  will  be  a  well-defined  matrix  of  cioaed-loop  tranafer  functiona.  We  can,  of 
courae,  compute  the  reault  directly  uaing  x  •  [B  *  aw]  x(o). 


Matrix  of  Cloaed-Loop  Tranafer  functiona 


a2  - 1 

a+ 1  5 

-4a*  1 

(•♦  D2" 

-5* 

•(■ ♦ I)  5* 

a(-4a  *  1) 

•(2a  -3) 

1" -18*  ♦  12  |  9a -6 

10a  —  50 1 

.a-1 

l  «g-«*5. 

.a2-**  1 

a  ♦  1  . 

L-56a*  14  !  15a  — 20 

60a-80  J 

k  a(#?t4) 

"-30  15  60  I 

-40  20  80  x,(0) 

(a  ♦  1) 


*(•*  » 4) 


26(a ♦  1) 


The  "reduced  atate"  an  aver  ia  obtained 
without  the  neceaaity  of  any  limiting 
procedsire  applied  to  the  feedback  gaina. 
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IMPLICIT  MODEL  FOLLOWING 


Observation 


W-H  approach  requires  no  modification  when  the  model  is  unstable. 


y  =  Hx 


x  *  nc  +  Gu  =>  X(a)  =  (Is-F)-1  GU  +  (is-F)-1  x(0)  =  AU  +  Bx(0) 

/*»  #  /*c+j« 

/  !  [y-Ijr],''.[y-Ly]  +U'RU  jdt  =  p-r  I  I  [(Is  -  L)Y]  :[(Is  -  L)Y]  +  U*RU  ds 

Jo  nj  Jc-i •>  * 

W-H  Equation 

[R  +  A#H#(ls-L).’?.(ls-L)HA]G  +  A<H#(ls-L)#a(ls-L)HBx(0)  =  % 

Example  1 


:  •■[:-:] 


H  =  I  =>  X(s)  = 


[:]  [-/ :] 

u(s)  +  /37'0(s  +  2) 


(s+  l)(s  +  2) 


[Is  -L] 


[■  "] 

—12  s  +  1  J 


X1  (0) 
Xj(O) 

x}(0) 


det[Is  — L]  =  s2  +  b-12 

=  (s*l>)(»“3) 


W-H  Equation: 

(a2-8-12)(ag  +  8-12)  [ap»  ♦  +  (Bg-s-l2)[-li>s-38  ;  sg  +  a-l2] 


(8+  lO(s  — 3) 

[1ka  +  38  ;  2s  +  14]  .  , 

U°  ■  WX(0)  *  W(-3)-  X(0) 


DD 


x(O)  =  iv 


X  -  [B  +  AW]x(0) 

Example  2 


["iV  *]  TXi 


(s 


J2 _ J  [X1(0) 

+  4)(a-3)  x2(0) 


Therefore,  K  ■*  [—11*  ,  -2] 


=  [is  -  Lj  x(O)  Check 


Same  as  Example  1,  except  R  =  0  Q  a  I  L 
Therefore, 


El 


8-2  1 
0  8-1 


[-12  6s] 

-f - -  x(0)  =s.  X  =  [0  -6] 

s  -  3a  +  2 


[18  -  L] 


U0  =  Wx(0) 


X  -  [B  +  AW]  x(0)  = 

ed  -  3s  +  2 

Therefore, 

Choice  of  ;  *  r  matches  only  the  poles. 


det[Is-L]  =  (s-l)(s-2) 


s  -  3  1 

r- 

-1 

-L^ - S-J  x(0) 

[is—  L]  *  = 

L  o 

8  -?  _ 

8‘  -  3s  *  2 


4  B  *  AW 
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APPENDIX  B 

SOLUTION  USING  SPECTRAL  FACTORIZATION 

The  regulator  W-H  equation  is 

[R  +  A#QA]U  +  A#QBx(0)  =  f  (B-1) 

Since  we  may  write 

[R  +  A#QA]  -  [I  +  A*K'3R[I  +  KA]  =  F#RF  (B-2) 

one  may  verify,  by  direct  substitution,  that 

U0  =  -F-1  R_1  [F^1  A*QBx(0)]+  (B-3) 

is  a  solution  to  Eq.  B-1.  This  solution  requires  only  that  R-1  exist. 

The  Wiener -Hopf  equation  for  the  filter-observer  problem  is 


[R+A*QA]Wa[<p^t  +HB^,B»HJ  +  =  *  (B-4) 

Setting 

cpvv'  +  =  (b-5) 

R  +  A*QA  =  F#RF  (B-6) 

A.«S».BA  -  K  <»-T) 

gives  the  following  form  for  Eq.  B-4; 

(F#RF)WaGG#  +  N  =  t  (B-8) 
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By  direct  substitution,  one  may  verify 


Wa  «  -F  1  R-1  [  F  J  N  G^1  ]  +  G-1 


(B-9) 


satisfies  Eq.  B-8.  That  is, 

-F*[f;1NG;1]  +  G#+  F.F^HG^G*  =  *  (B-10) 

F*{-*F*1»G*1]+  +  =  f#{f;1ng;1}_g^  (B-11) 


Equation  B-11  satisfies  the  W-H  requirement  of  forcing  to  consist  of 
time  functions  which  exist  only  for  negative  time. 
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APPENDIX  C 

A  VARIATIONAL  METHOD  FOR  DERIVING  W-H  EQUATIONS 

Suppose  the  integrand  of  the  performance  index  (or  a  particular  element 
of  the  index)  has  the  form 

<S>  =  X*H*HY  (C-1) 

the  first  variation  is  found  by  taking  the  gradient  with  respect  to  H*: 

V®H*  =  x;[HY]’  =  Xf  (— s  )Y'  (— s  )H'  (s )  (C-2) 


Example  1 : 

From  Section  II,  the  regulator  problem  gave 
$  =  X*QX  +  U*RU 

with 


Therefore, 


X  =  AU  +  Bx(0) 

<j>  =  [U*A*  +  x'(0)B*]  Q  [AU  +  Bx(0)]  +  U*RU 


(C-3) 


(C-U) 

(C-5) 


Taking  the  gradient  with  respect  to  U*  gives: 


V<pU*  =  [A*]*[Q(AU  +  Bx(0)]»  +  (RU)’ 

Take  the  transpose  and  write 

[R  +A*QA]  U0  +  A*QBx(0)  =  \Jf 


(C— 6) 


(C-7) 


verifying  Eq.  6. 


0NR-CR21 5-260-1 


0NR-CR21 5-260-1 


If  V  and  N  are  independent,  Eq.  (C— 1 3 )  reduces  to 
Waf<%»  +  -  Bcpj^,B*H*  =  * 

This  verifies  Eq.  42. 

Example  3: 

Equation  67, 

$  =  X*QX  +  U*RU 


with 

U  -  Wa[V  +  HBN] 

X  --  AWa[V  +  HBN]  +  BN 

becomes 

«  r  [N*B*  +  (N*B*H*  +  V*)Wa*A*]Q[AWa(V  +  HBN)  +  BN] 
+  [N*B*H*  +  V*]  Wa#R  Wa  [V  +  HBN] 

Therefore 

-  [N*B*H*  *  v.r  j  [A*Q(AWa(V  +  HBN)  +  BN)]' 

+  fRWa(V  +  HBN)]'  J 

Again,  we  prefer  to  work  with  the  transpose 


TR-1 12^-1 


C-3 


(C-14) 


(C— 15) 

(C- 16) 
(C-17) 

(C-18) 

(C-19) 
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^a*^  =  {  A*QAWafV  +  HBK]  +  A*QJBN  +  RWjV  +  HEN]  |  [H*^  4  V*] 

=  [B  +  A*QA]WJV  +  HBN][N*B*H*  +  V*J  +  A#QBNfN*B*H#  4  y# 

Given  'Pyjj'  =  <PjJy'  =  0,  then  taking  the  expectation  yields 

[R  +  A*QA]Wa[q^f  +  HB^,B*f£*]  4  A^QBp^.BiH*  =  * 

verifying  Eq.  71 . 


fC-20) 


(C-fcl) 


(C-22) 
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APPENDIX  D 
THE  OUTPUT  REGULATOR 

The  system  is  linear  and  time  invariant; 

x  =  Px  +  Gu 
y  =  Hx 

Transforming  Eq.  D-1  gives 

X  =  [is  -  F]_1  GU  +  [Is  -  F]“ 1  x(o) 

Y  =  HX 

The  block  diagram,  assuming  a  control  law, 

y  =  — KX  (K  may  be  frequency  dependent) 

is  shown  in  Fig.  D-1 . 

From  Fig.  D-1,  write 

U  =  — [I  +  KHA]-1  KHBx(0)  =  WaHBx(0) 


(D 


(D- 


(15- 


Figure  D-1 .  Closed-Loop  Configuration 


TR- 1125-1 


D-1 


0NR-CR21 5-260-1 


Minimizing  the  quadratic  index, 


J 


1 

2«j 


(X^QX  +  U#RU)  ds 


X#  =  X'(-s) 


(D-U) 


with  respect  to  Wa  gives  the  W-H  equation  as: 

[R  +  A#QA]Wa(HBx(0)x'(o)  B*Hj  +  A  QBx(O)  x’(0)B#H#  =  *  (D_5) 

Equation  D-5  can  be  solved  using  the  direct  approach  or  spectral  factoriza¬ 
tion.  Using  the  factorization  approach,  write  Eq.  D-5  as 


[F#RFjWaGG*+  N  =  ,!•  (D-6) 

so  that 

wa  a  -F"1  R“T  [f"1  NG~1]  +  G~t  (D-T) 

F  can  be  computed  using  the  "full  state"  feedback  regulator  gains; 


F  =  I  +  KA 


for  example: 


x 


Therefore, 


X(s) 


(D-8) 


(D-9) 


(D-10) 
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